Skip to content

Pin Python dependencies and install them in a virtualenv #7404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

damz
Copy link
Contributor

@damz damz commented Jan 22, 2017

(Alternative version from #7403)

Installing sphinx started to fail about a week ago, with pip incorrectly trying to remove a Python package shipped at the system level in Platform.sh environment.

It looks like it started failing because of a bug in the older version of pip that ships with Debian Jessie. We (Platform.sh) install global Python dependencies as user-site packages (using pip --user). We also ship system-site packages that we use for our own benefit.

So two issues here:

  • First, that older version of pip. We are going to look into upgrading it (a new batch of images based on Debian Stretch are cooking);
  • The build dependencies of the Symfony Docs are not pinned, which makes the build non-repeatable. I'm guessing that in this case it broke when jinja2 ended up requiring markupsafe>=0.23 (but that's just a guess).

Fix the issue for good by installing in a virtualenv and pinning all the dependencies via pip freeze.

@javiereguiluz
Copy link
Member

Amazing! @damz thanks so much for looking into this. Now I'm sure I would have never fixed it 😅

Installing `sphinx` started to fail about a week ago,
with `pip` incorrectly trying to remove a Python package
shipped at the system level in Platform.sh environment.

It looks like it started failing because of a bug in
the older version of `pip` that ships with Debian Jessie.
We (Platform.sh) install global Python dependencies as
user-site packages (using `pip --user`). We also ship
system-site packages that we use for our own benefit.

So two issues here:

 * First, that older version of `pip`. We are going to
   look into upgrading it (a new batch of images based
   on Debian Stretch are cooking);
 * The build dependencies of the Symfony Docs are not
   pinned, which makes the build non-repeatable.
   I'm guessing that in this case it broke [when jinja2
   ended up requiring markupsafe>=0.23][jinja2-commit]
   (but that's just a guess).

Fix the issue for good by installing in a `virtualenv`
and pinning all the dependencies via `pip freeze`.

[jinja2-commit]: pallets/jinja@5453db1
@damz
Copy link
Contributor Author

damz commented Jan 22, 2017

You can pick either this one or #7403, and it is purely a "pick your poison" question:

  • This one you will have to update the pinned dependencies from time to time, but the build is repeatable;
  • Move python dependencies inside a virtualenv #7403 you don't have to update the dependencies (you get updated versions automatically), but that might make the build randomly fail.

@damz damz force-pushed the pr/pin-dependencies branch from 23f79c3 to c025812 Compare January 22, 2017 22:06
@javiereguiluz
Copy link
Member

We've finally decided to go for this version, so we'll close #7403 and we'll merge this soon. @damz thanks for your great help solving this issue!

@xabbuh
Copy link
Member

xabbuh commented Jan 27, 2017

Thank you so much for fixing our platform.sh builds @damz! Happy to merge it.

@xabbuh xabbuh closed this in 8146f89 Jan 27, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants